Radio spectrum is scarce and the FCC, DoD and other international spectrum management organizations are constantly looking for ways to more efficiently utilize this limited spectrum. Demand for spectrum is continuing to rise due to the explosive growth of data, voice, messaging, and video applications. One solution to meeting the need for improved spectral efficiency as measured by bits/Hz/user is adaptive radios (also referred to as dynamic spectrum access (DSA) or cognitive radios (CR)). Adaptive radios can change their transmission characteristics to maximize transmission capacity and coverage while conserving spectral usage.
One of the challenges of deploying adaptive radio technology is that it cannot be fielded without comprehensive testing, and it cannot be tested in a densely RF system populated, live environment for fear of potentially interfering with existing spectrum users (primary users). Field testing is preferable to lab testing but requires a realistic environment where it can be verified that the System under Test (SUT) will not interfere with primary users or other spectrum users. Laboratory testing is more cost and schedule effective, repeatable, controllable and observable, but generally lacks in realism, especially with respect to RF environmental considerations.
There is an established and growing need to comprehensively test and evaluate performance of these new adaptive devices/systems in known and postulated environments to establish behavior characteristics (“average behavior”) and reduce unintended field behavior risk (“abhorrent” or “rare event” behavior). Traditional test methods are increasingly stressed by the proliferation and diversity of the devices/systems and operating environments. As used in this application, RF system include RF devices, such as a transmitter, receiver or navigation device, as well communication systems, navigation systems, radar systems or other systems using transmitters or receivers. As used in this application, testing means evaluating input and output parameters for an RF system across a parameter search space in an RF environment in order to determine the behavior characteristics of the RF system. Historically, RF system testing has fallen into two broad categories, field testing and laboratory simulation/testing. Field testing as illustrated in FIG. 1 involves placing some number of devices in a realistic field environment and exercising them to test performance against specified functionality. Full-featured field tests place the wireless transceivers in a field scenario containing some representative RF environment where they will be operated while test data is collected. These sorts of tests are often expensive and complex to orchestrate, and can lack flexibility since mixes of test transceiver numbers/types/locations, incumbent RF user numbers/types/locations and RF propagation conditions cannot be systematically varied to collect comprehensive data. FIG. 1 schematically depicts a typical field test equipment setup. Wireless Transceiver Units Under Test (UUT) 100 operate in some RF environment 110. The RF emissions are subject to the noise, path loss, multipath transmission and interferers found in the local RF environment. Test instrumentation 120 is established to measure the performance of the UUT and other PU of the RF environment. In order to accomplish a field test of this variety, the UUT must be physically located in the test RF environment, and test instrumentation must be constructed. In order to vary the numbers/types/locations of UUT and PU, physical units must be acquired and placed in the RF environment. In order to vary the RF environment, different field venues must be available. Additionally, test instrumentation must be provided and adapted for each UUT/PU/test environment scenario where testing is to be accomplished.
Many factors must be considered when selecting and configuring the field test area including the specific type and host platform for the Systems Under Test (SUT), the characteristics and quantity of other RF interferers in the environment, and environmental factors that affect the radio propagation including terrain and morphology. Field test methods have been viewed as the most realistic, but many growing challenges limit their ability to be compelling. These challenges include:                Difficulty and complexity in testing high platform dynamic systems        More devices/systems to test        More functionality & complexity to test including adaptive/cognitive behavior        Test ranges require a broad set of realistic physical layouts        Requirements to emulate location-specific RF environments including propagation and interferers        Requirements for conditions not realizable on test ranges including prohibition by FCC rules        RF environment control difficult due to encroachment of commercial RF sources.All of above lead to increased costs, longer schedules, more requirements on field test assets and ranges, and potentially lower confidence in results. For adaptive RF systems, field testing is not practical. Laboratory test methods are generally more cost and schedule effective, are more controllable and observable, but generally are lacking in realism, especially with respect to RF environmental considerations.        
There exist many variations of lab testing approaches, but they can be generally bounded by “RF Path Simulator” and “Software Modeling” variants. The RF Path Simulator approach shown in FIG. 2, which interconnects RF systems/devices with conventional laboratory test equipment such as signal/noise generators, is only applicable to simple RF environments, small numbers of devices/systems under test with simple antenna systems, and small number of primary users/interferers. Lab-based testing using cable-based interconnection for RF emissions of UUT and the RF environment is a prior art approach to testing to overcome the challenges of placing and monitoring devices in the field environment. FIG. 2 depicts a typical lab-based equipment setup. As in field testing, Wireless Transceiver Units Under Test (UUT) 100 are acquired and instrumented with Test Instrumentation 120. Instead of the RF environment being that found in the field, RF test equipment such as signal generators are used to produce Interferers 210, Noise Generators 220, and Path Simulators 200 to simulate path loss and multipath in an RF channel. RF Interconnection 230 is accomplished using RF cables such as coaxial cables. This test set up approach reduces some of the complexities of field testing, but introduces new concerns over RF environment realism. Further, it still requires the physical introduction of new UUT and RF test equipment into the configuration for comprehensive transceiver configuration and RF environment results.
Traditional software modeling approaches as shown in FIG. 3 have historically made simplifications about the physical environment/radio propagation effects, and generally cannot support any hardware in the loop (HITL) test cases. Their validity is therefore limited to a narrow group of test cases and not well suited to the adaptive RF system test problem. A variation on RF cable-connected lab testing has become more prevalent and straightforward as wireless transceiver devices have tended towards digital waveforms and digital hardware or software implementation. FIG. 3 depicts a typical framework for modern wireless communications devices as defined by the prior art OSI model. Here, different functions in the Wireless Transceiver 100 are allocated to layers in the functional stack 300. The physical layer in stack 300 is where the waveform-related functionality is contained. The physical layer can be segregated into a digital implementation portion 310 and an analog portion 320. Typical functions in the digital transmit portion 310 are waveform generation and digital to analog conversion. Typical functions found in the analog portion 320 are baseband to RF conversion. Other digital processing functions associated with non-physical layers (2 through 7) are performed through digital data processing blocks 330.
A laboratory-based testing approach that combines the advantages of true RF path/environment emulation and HITL, but implemented in the digital domain under software control, has the potential to deliver the advantages of the different lab methods with the realism of field testing. The test platform disclosed in commonly owned U.S. patent application Ser. No. 12/787,699, titled “Wireless Transceiver Test Bed System and Method”, which is hereby incorporated by reference, follows this approach. The present disclosure adds improvements directed to a method to control the RF environment to execute a sufficient number of test cases for validity and schedule the test cases so that a limited number are required for execution. This facet of the test bed problem is further described below.
Perhaps the most challenging part of adaptive RF system testing is addressing the vast number of test cases that may have to be scheduled to comprehensively test an adaptive RF system. To illustrate the magnitude of the problem, an example test scenario for an adaptive navigation receiver is presented. The test conditions can be grouped into 5 categories, each with a large number of individual parameters as follows:                1. GNSS Signals (# systems, # satellites, positions of satellites, status of satellites (i.e. health, accuracy of correction data, etc.))        2. Interference Signals (#, type, position, characteristics)        3. Augmentation (existence, types, characteristics of types (including the following))                    a. Other RF Source Augmentation (i.e. Signals of Opportunity)            b. Mechanical Augmentation (i.e. IMU)            c. Correction Augmentation (i.e. WAAS)            d. Assist Augmentation (i.e. A-GPS)            e.                        4. Propagation Environment (GNSS to PNT, Interference to PNT, Augmentation to PNT (if applicable))        5. PNT System Configuration (host platform considerations (varies by host platform)), orientation to sources (up to 6 degrees of freedom), # RF channels, antenna systems, user configurable parameters.        
It can be easily envisioned that the number of test cases could routinely reach into the millions (or higher for more complex RF system types). Two challenges result from this condition. First, the time and associated cost of performing the test may be prohibitive. Second, the vast amount of data produced by comprehensive testing may make useful conclusions about the performance difficult or impossible to formulate. A desirable capability of the test asset would be a test methodology that significantly reduced the number of tests run while maintaining the validity of the data (the ability to extract the performance characteristics of the RF system under test).
Based on a review of the available RF system test beds that exist in industry and academia (including those referenced in U.S. patent application Ser. No. 12/787,699), a wireless transceiver test bed approach, capable of efficiently producing valid performance test data, and yet is scalable, flexible and affordable is not known.
The present disclosure utilizes emerging technologies and trends in the areas of optimal search algorithms, digital signal processing, wireless device design, wideband networks, computer and software architecture/capability and software-based modeling to provide a means to address these shortcomings. Specific technology innovations that contribute to various aspects of the present disclosure include:                digital signal processing power and available algorithms and models        ability to digitize RF with high fidelity        emerging software defined radio (SDR) software architectures, such as SCA (Software Communications Architecture)        emerging commercial off-the-shelf digital radio and SDR components (hardware and software)        ever increasing broadband connectivity between distributed sites        comprehensive and advanced RF propagation models        RF emitter models being built in software        proliferation of radio functionality being digital and implemented in software with discrete events (bits, bursts, frames, etc.).        standardization of baseband digitized interfaces to SDRs (such as the VITA-49 Radio Transport Protocol)        optimal search algorithms including multi-queue branch and bound algorithms.        
The present disclosure is not limited to adaptive wireless devices in the application area of communications, but broadly applies to all wireless devices and networks including receive only, transmit only and diverse applications such as sensing, radar, and jamming. Further, it is not limited to testing adaptive RF systems and could also be used to automatically test conventional RF systems. The same properties of effectively and efficiently testing apply in that the test system would automatically produce valid results using a reasonable number of RF environment scenarios.